Data replication is a technique used to maintain copies of data at separate locations. For example, data can be replicated on several different sites within a corporation's campus and/or on several different ones of the corporation's campuses. If the data is replicated at different sites, and if the failure of the systems storing the data at one site is unlikely to cause the failure of the corresponding systems at another site, replication can provide increased data reliability. Thus, if a disaster occurs at one site, an application that uses that data can be restarted using a replicated copy of the data at another site.
The system (e.g., the host) that consumes and/or modifies the data that is being replicated is referred to as the production system. For example, a host implementing a file system is the production system with respect to the data organized by the file system.
Typically, replication activities fall into one of three categories: database replication, file-level replication, and volume-level replication. Database replication is limited to use with databases. File-level replication is based on tracking changes to the data at the file level. Whenever a modification to a file is detected (e.g., by the file system or a filter driver), that modification is replicated to the secondary site. One drawback to file-level replication is that typically, both the primary and the secondary site need to use the same file system in order for replication to be successful. Another drawback is that the replication must be initiated by the production system. Accordingly, the functionality needed to initiate replication consumes resources at the production system. This can interfere with the ability of other applications executing on the production system to provide services to clients and/or users in a timely and efficient manner.
In contrast to file-level replication, volume-level replication tracks changes at the volume level. Replication can be performed on data volumes by designating one volume as the primary volume. One or more secondary volumes are then synchronized with the primary volume. These secondary volumes can each be located at a different secondary site. Initially, a secondary volume can be synchronized to the primary volume by copying all of the data on the primary volume to the secondary volume. The data on the primary volume can be copied to the secondary volume by transferring all the data over the network, by creating a backup of the primary volume and restoring the secondary volume from the backup, or by attaching one or more mirrors of the primary volume to the secondary volume. Replication then continues by propagating any changes to data in the primary volume to the secondary volumes.
One drawback of volume-level replication is that all changes to data within the volume are replicated, even if it is not necessary to replicate some portions of the volume. For example, it may not be necessary to replicate changes to file system metadata stored on the volume. Similarly, in a software development environment, it is not necessary to replicate executable files as long as the corresponding source files (which can be used to regenerate the executable files) are replicated. However, since changes are tracked at the volume level, there is no way to differentiate between file data and file system metadata or between different types of files, and thus changes to both types of data and/or both types of files will be replicated. Another drawback of volume-level replication is that a small file-level change can cause an entire page to be flushed to the volume, which will in turn cause the entire page to be replicated to the secondary site. This unnecessarily increases the amount of data transfer on the network.
As the above examples show, there are potential drawbacks to using file-level, database replication, and volume-level replication. Accordingly, new techniques for replicating data are desired.